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A subscriber characterization and advertisement monitoring system is presented in which subscriber viewing habits are monitored 
to determine demographic profiles. These profiles can be utilized for the matching of advertisements to subscribers based on 
their viewing habits and estimated demographics and product interests. The system can be run locally in a television set-top 
(1808) or can be run in client server mode where channel selections are transmitted from the residence (1800) to a centralized 
switching location server (1840) such as a telephone office or Internet Service Provider. In client-server mode the channel 
selections are monitored at the centralized location (1840) which also performs the subscriber characterization. The system also 
provides the ability to monitor if advertisements were viewed and for what duration. 

(57) Abrege 

L'invention concerne un systeme de determination de profil d'abonne et de surveillance publicitaire (100), dans lequel on 
controle les publicites qu'un abonne regarde, afin de determiner des profils demographiques. On peut utiliser ces profils pour 
faire correspondre les publicites destinees a des abonnes en fonction de ce qu'ils regardent, revaluation des donnees 
demographiques, et I'interet presente par un produit. Le systeme (100) peut etre mis en application localement dans un appareil 
de television (1808), ou peut etre utilise en mode serveur-client, les selections de chaines etant transmises de la residence 
(1800) vers I'emplacement de commutation (1840) centralisee (serveur), tel qu'un central telephonique ou un fournisseur de 
services Internet. En mode serveur-client, les selections de chaTnes sont controlees au niveau de I'emplacement de commutation 
(1840) centralise, ce mode determinant egalement le profil d'abonne. Le systeme permet egalement de controler si les publicites 
sont regardees et pendant combien de temps. 
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(57) Abstract 

The present invention relates to an interactive digital system (1) enabling viewers full and active participation in experiencing a 
live broadcast event. Particularly, the presentation of the live event is personalized for the viewer through the provision of options, 
including multiple video streams, associated with different camera angles, for example, and integrated audio and graphic segments. Further, 
information obtained from related Web sites can be integrated into the live program. 
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BACKGROUND OF THE INVENTION! 

Interactive video and audio presentation systems are currently being 
introduced into the entertainment and educational industries. A prominent 
interactive technology that has been applied successfully in these industries is 
based on providing interactivity in a one-way system through the provision of 
multiple time-synched parallel channels of information. For example, 
commonly owned Freeman et al. patents, U.S. patent nos. 4,264,925 and 4,264,924, 
which provide both audio and video interactivity, disclose interactive television 
systems where switching among multiple broadcast or cable channels based on 
viewer selections provides an interactive capability. 

These systems have been enhanced to include memory functions using 
computer logic and memory, where selection of system responses played to the 
viewer are based on the processing and storage of subscriber responses, as 
disclosed in Freeman patent, U.S. patent no. 4,507,680. 
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The benefits of providing interactivity through the use of different audio 
responses is disclosed in Freeman, U.S. patent nos. 4,847,698, 4,847,699 and 
4,847,700. These television systems provide a common video signal accompanied 
by several synchronized audio channels to provide content related user selectable 
5 responses. The audio signals produce different audio responses, and in some 

cases, these are syllable synched to a first audio script and to the video signal (such 
as to a person or character on a display), providing the perception that the 
person's or character's mouth movements match the spoken words. 

Interactivity is brought to the classroom in the Freeman U.S. patent no. 

10 5,537,141. The distance learning system claimed in this application enhances the 
classroom educational experience through an innovative use of interactive 
technology over transmission independent media. When an instructor, either 
broadcast live on video or displayed from videotape, asks a question, each and 
every student responds, preferably by entering a response on a remote handset, 

15 and each student immediately receives a distinct and substantive audio response 
to his or her unique selection. The individualization of audio response from the 
interactive program is a major aspect of the invention. 

Individualization of audio is brought to the home based on the technology 
disclosed in Freeman U.S. patent no. 5,585,858. This system provides a program 

20 that can be watched on any conventional television set or multimedia computer 
as a normal program. But if the viewer has a special interactive program box 
connected to the television, he or she can experience a fully functional interactive 
program. Each interactive viewer enjoys personalized audio responses and video 
graphics overlayed on the screen. The interactive program can be provided to 
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television sets or to computers by cable, direct broadcast satellite, television 
broadcast or other transmission means, and can be analog or digital. 

However, what is needed is an interactive presentation system for 
providing true video, audio and graphics interactivity with digital programs 
5 broadcast live. Such a system must efficiently package all the digital elements of 
the live interactive program at a centralized control studio and allow viewers at 
home to receive personalized interactive programming. 

SUMMARY OF THE INVENTION 

10 In accordance with the invention, there is provided an interactive digital 

system allowing the viewer active participation in selecting digital video streams, 
associated with different camera angles, for example, and integrated audio and/or 
graphics segments. Further, Web pages from Internet Web sites can be integrated 
into the program. The invention is particularly suited for the environment of 

15 live events, such as the broadcast of live sporting events. The viewer can appear 
to direct the camera shots by instantly changing among various camera angles, 
choose player interviews, or display associated statistical data on the team or 
players via graphics. In this manner, the system allows the individual subscriber 
to act as if he or she has control over how the program is directed and presented 

20 on their personal television set. In addition to selecting different camera angles, 
various audio options, closeups, slow motion, replays, graphics overlays, graphics 
or audio from Web sites, etc., are all possible. Further, games can be integrated 
with the live sports programming to increase viewer interest. 

Thus, viewers can customize the content of programs. The interactive 

25 digital programming of the present invention is particularly advantageous for 
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viewing live sporting events. Viewers are not limited to selecting from multiple 

camera angles, but may also call up player statistics on demand, listen to selected 

player interviews, etc. Cameras can be focused on different segments of an event. 

Further, video options could include video replay, slow motion effects, isolation 
5 on a particular player or group, etc. Changes are seamless, thereby adding to the 

effect that the viewer is directing the television show just as a director now does 

from a control room. 

This "director" role by the viewer is possible due to the interactive 

technology of the present invention and also due to the digital compression and 
10 transmission scheme which allows for much greater information throughput 

over a given bandwidth, allowing viewers to choose from angles that are already 

available but presently cut by the director. 

The digital interactive system is based upon branches which occur in the 

course of the full-motion video. Branches are real-time parallel paths that may 
15 be other full-motion video segments, graphics which are integrated into the 

video, audio segments, and/or retrieved Web pages which are integrated into the 

live event. 

Sometimes, the interactive digital system will act upon the viewer's 
response immediately; other times, it will utilize ACTV's unique "profiling" 

20 concept to act upon the response later. This technology enables the system to 
"remember" the viewer's responses and desires, and integrate them into the 
video, audio, graphics and/or Web site information at a later point. For example, 
the viewer could specify at the beginning of a football game to isolate the 
offensive quarterback of a particular team. Thus, whenever the team of choice is 

25 on offense, the video isolation of the quarterback is displayed to the viewer 
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automatically. Or, based on how a viewer has selected camera angles, replays, etc., 
over the past five minutes, the system acts to mimic these selections at later times 
during the program. The system of the present invention "learns" from the 
viewer how they want to view the game, and thus, continues viewer selection 
5 sequences made earlier. 

At the source, the present invention comprises a plurality of video 
cameras, each of the video cameras relaying a different predetermined view of an 
event. The video signals corresponding to the different cameras are forwarded to 
a central control studio. Further, one or more audio signals or graphic statistical 

10 overlays can be collected and sent to a central control studio. After receiving the 
video, audio, and graphics signals at the central control studio, these signals are 
digitized and compressed in digital video and audio compressors. These signals 
are then combined with special data codes into a "digital package," and 
subsequently, transmitted over a cable distribution system. The special data codes 

15 are the keys to unlocking the interactive potential of the program. 

The digital program signals are transmitted to a receive site by any suitable 
transmission means. Once received by a receive antenna, the digital program 
signals are passed along on a digital cable television distribution system to the 
viewer homes. Further, some other signals or commercials can be inserted at the 

20 local head end. The signals are received and processed in a digital cable box. 

Selections of the video, audio, graphics display and /or Web pages can be made as 
a function of immediate viewer entries, or to interrogatory responses presented at 
the beginning or during the program, or based on a prestored viewer profile. 
Once a decision is made to switch from one video option to another video 

25 option, the digital switch is performed seamlessly. 
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As mentioned above, the program at predetermined times or immediately 
upon user entry can retrieve and branch to informative segments from Web sites. 
For example, a viewer watching a sporting event, through the system of the 
present invention, can receive a stream of Web pages which provide additional, 
5 specific information relating to a favorite player, team or perhaps the remaining 
schedule for the sports team, as examples. In addition, users can take advantage 
of the two-way capabilities of the Internet to respond to polls or to link to 
additional sites. 

Another Internet-based application allows advertisers to speak more 
10 directly to consumers by directly sending Web pages to the consumer instead of 
merely displaying Web addresses in their commercials. The particular 
advertising information from Web sites can be targeted to viewers based on the 
viewer profile, stored either in the digital set top box or at the cable headend. 
Alternatively, Web site access can be initiated by the viewer by simply clicking on 
15 the remote during the commercial. Thus, viewers have the capability to 

individually select Web sites if they want more information from advertisers, for 
example. 

The video programming and corresponding Internet pages can be viewed 
either on personal computers equipped with a television card on special digital 
20 cable boxes with stored interactive Internet application software providing 
Internet access, or on digital television sets, all of which would utilize the 
specialized TV/Internet software of the present invention. 

The present invention also has applications for other types of 
programming. For example, viewers can direct the scenes of a murder mystery. 
25 Switching from one scene to another can be done seamlessly without noticeable 
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effect on the viewer. Further, the present invention can be used for any kind of 
live or pre-recorded event. For instance, a music concert or a political 
convention can be enhanced in the manner of the invention. 

Accordingly, a primary objective of this invention is providing an 
5 enhanced digital live program allowing the display to be tailored to the user's 
desires, choices or interests. 

It is an object of this invention to personalize and enhance live sporting 
events for the viewer. 

10 DESCRIPTION OF THE DRAWINGS 

Figure 1 is a diagram of the network and equipment for providing live 
digital programming. 

Figure 2 is a block diagram of an interactive digital cable box allowing 
seamless switching between video signals. 
15 Figure 3 is a block diagram of an alternative dual-tuner interactive digital 

cable box allowing seamless switching between video signals. 

Figure 4 is a block diagram of another alternative interactive digital cable 
box allowing seamless switching between video signals. 

Figure 5 is a time diagram showing a representation of trigger points and 
20 corresponding video, audio and /or graphics segments, one or a combination of 
which are selected for presentation to the subscriber immediately after the 
execution of the trigger point function. 

Figure 6 is a block diagram of an alternative embodiment of the interactive 
system including Internet access. 
25 Figure 7 is a block diagram of the two-way configuration of the system. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 
The present invention is an interactive digital system 1 for producing a 
powerful personalized program allowing the home viewer an expanded set of 
programming options. Digital TV streams are put into digital packages made up 
of video, audio, data codes and graphics, and are used to provide personalized 
responses to viewer selections. Such responses can be further enhanced by 
allowing access to Internet Web sites 170. In this manner, sports such as golf, 
football, baseball, basketball, etc. can now be watched with greater interest and 
involvement. However, even further enhanced interactivity is possible with the 
present invention due to the provision of various profiling and memory 
features. 

As shown in Figure 1, the present invention begins with the gathering of 
several possible video streams by way of cameras 100, strategically located at a 
sporting event 10, for example. Currently, many cameras 100 are employed at a 
sporting event 10. Super Bowl coverage, for example, typically encompasses 25 to 
30 cameras. These live video streams can be integrated with recorded video 
streams which, for example, could include highlights from the current game or 
past games, player profiles, etc. To describe the components and operation of the 
present invention, the production and transmission of a live sporting event 10, 
i.e., football, is chosen to present the invention features. However, other 
applications can be appreciated by the reader, including several disclosed below. 
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A. The Control Studio 

The interactive broadcast program is prepared at the control studio 5 into 
digital packages. The control studio 5, as shown in Figure 1, allows a producer to 
create and introduce interactive elements during a live broadcast. In one 
preferred embodiment, the producer pre-records a set of interrogatories or 
instructions for the user. These interrogatories may include such questions as the 
following: 



SELECT THE CAMERA ANGLES/OPTIONS THAT YOU PREFER: 

OPTIONS: END-ZONE 

FIFTY YARD LINE 
FOCUS ON THE QUARTERBACK 
FOCUS ON THE DEFENSE 
FOCUS ON THE CHEERLEADERS 
PROVIDE HIGHLIGHTS 

WHO IS YOUR FAVORITE TEAM IN THE GAME? 

OPTIONS: BLUE TEAM 
RED TEAM 

WHO IS YOUR FAVORITE PLAYER IN THE GAME? 

OPTIONS: RED RUNNER 

BLUE QUARTERBACK 
BLUE DEFENSIVE END 
RED QUARTERBACK 
ETC. 

ARE YOU INTERESTED IN RECEIVING STATISTICAL SUMMARIES? 

WOULD YOU ENJOY LISTENING TO PLAYER INTERVIEWS DURING 
THE GAME? 

Such interrogatories can be presented to the viewer at the beginning of the 
broadcast or scattered throughout the program. Interactive responses to such 
interrogatories would include video, audio and graphics personalized to the 
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particular viewer. The preparation of the graphics for presenting such 
interrogatories occurs off-line at the control studio 5 using chyron or any graphics 
language. These interrogatories or instructions will ultimately be displayed to the 
home viewer, preferably in the form of graphics, to facilitate the interactive 
5 responses. The producer creates these graphic video slides of questions on a 
computer using the text editor and chyron. Associated with each question, the 
producer enters a number of possible viewer options. Then, the producer relates 
each possible user entry to one or more corresponding interactive responses. If 
the response is information from an Internet Web page, the producer will 

10 indicate the particular Uniform Resource Locator (URL) of the Web page. The 
producer sets a time stamp for when each particular question will appear during 
the program. As explained below in more detail, the viewer response to a query 
will be used to direct which video (and/or audio, graphics, or Web page) option 
will be provided to the viewer. In the present invention, the interactive response 

15 to the query can occur immediately following the entry of the viewer entry or at 
some predetermined later time in the program using "trigger points," 500 as 
explained in detail below. 

With respect to the video segments, cameras 100 are preferably trained on 
different segments of the sporting event 10. As is common with broadcasts of a 

20 football game, for example, cameras 100 could be located in the endzone, press 
box, the field and at various other locations throughout the stadium. Further, 
various video options can be created including video replay, slow motion, 
isolation on cheerleaders, particular player or group of players. Instant replays are 
created by delaying the live feed for a certain number of seconds. These video 

25 streams are sent to a control studio 5. The control studio 5 contains the necessary 



WO 98/41020 PCT/US97/09045 
11 

equipment for packaging the program for delivery to the viewers. The studio 5 
contains a video switcher 105 which receives the live signals from the cameras 
100 by way of various input lines. Further, lines carrying recorded video streams 
from one or more VCRs 110 , computers or CD players feed into the video 
5 switcher 105. The video switcher 105 also receives video inputs from the control 
computer 135. Further, various graphics screens, depicting, for example, sports 
team or player statistics can be designed with the control personal computer 135 
and forwarded to the digital video switch 105. The producer, via the control PC 
135, directs which video options to pass through the video switcher 105. At the 

10 output of the video switcher 105, each of the different output video streams access 
a separate encoder 125 and are all GENLOCKED, so that each video stream is 
synchronized with the other video streams. 

After encoding, the video streams are input into a video compressor 125. 
Preferably, the digital compression scheme is MPEG-2. Preferably, 64 Quadrature 

15 Amplitude Modulation (QAM) is used as the modulation scheme. In this digital 
embodiment, four channels of digitally-compressed video content would carry 
about 27 Mbps using 6 MHz of bandwidth. Alternatively, if 256 QAM is 
employed, seven-to-one video compression can be achieved with the MPEG-2 
scheme. While MPEG-2 is the preferred compression scheme, the signals can be 

20 compressed according to any known standard including MPEG-1, JPEG, or other 
DCT coding scheme, wavelets, fractals or other transform or waveform based 
technique. 

The control studio 5 also contains an audio switcher 115 which receives 
live audio signals from microphones or recorded audio from tape players 120, 
25 CDs, VCRs 110, etc.. The control computer 135 sends commands to the audio 
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switcher 115 directing which audio options should pass through the switch 115. 
Further, in the audio switcher 115 the various audio signals can be aligned to 
match the various video signals in time. In addition, VCR audio output is 
received by the audio switcher 115. The present invention can accommodate any 
5 number of audio signals as output from the audio switcher 115, as directed by the 
producer. The audio outputs are received by an digital audio 
encoder/compressor 130. The audio signals are then preferably sampled, encoded 
and compressed in the digital audio encoder/compressor 130. The encoding 
technique can be a waveform coding technique such as PCM, ADPCM or DM. 

10 Alternatively, the signals can be encoded using synthesizer or vocoder techniques 
such as MUSICAM, Linear Predictive Coding (LPC), Adaptive Predictive Coding 
(APC), and Sub-band coding. Generally, the transmission rate is about 256 kbps 
per audio for the stereo pair. 

The timing and control for integrating the various multimedia elements is 

15 provided by the ACTV authoring language, a unique set of interactive data codes 
to facilitate the interactive process. The data codes are stored in memory in the 
control computer 135 as part of the ACTV programming language. The codes 
comprise commands, or branch codes, for branching between interactive options, 
timing signals for controlling the interactive program, data or text, commands for 

20 termination and initiation or interactive program viewing, or triggers for 
executing macros. Preferably, these commands are output from the control 
computer 135 and multiplexed with the video streams in the MPEG-2 compressor 
125, as shown in Figure 1. Interactive options that can be branched to based on 
the branch codes include video segments, audio segments, graphics segments 

25 and/or identified Web pages. 
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There are several commands in the ACTV authoring language that 
function to provide the interactive sports programming applications. These 
commands are explained below in Section D, which details programming 
applications. 

5 

B. The Transmission System 

As shown in figure 1, the digital interactive system 1 uses an interactive 
program delivery system with any transmission means including satellite 15, 
cable 150, wire or television broadcast 175 to deliver the interactive program 

10 (hereinafter "composite interactive program") from the control studio 5 for 
distribution to subscribers in their homes. At the control studio 5, the signals 
from the digital multiplexer 140 are converted to RF and distributed to a 
microwave 175, cable 150 or satellite 15 network. Preferably, the digital interactive 
signal is forwarded from the control studio 5 to a cable headend 150, and 

15 subsequently, sent to the homes via the cable network. 

The program is preferably the broadcast of a live event. For example, live 
sporting events with added interactive elements can be broadcast from the 
control studio 5. Such live interactive elements could be different camera angles 
100, slow motion video, etc., as discussed above, while also incorporating 

20 prerecorded interactive segments such as highlights. Alternatively, the program 
can be produced off-line and stored in a program storage means at the control 
studio 5. 

In a satellite broadcast transmission, the digital interactive signals are 
transmitted to uplink equipment where they may be multiplexed, upconverted, 
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modulated, amplified and transmitted by satellite 15 to the receiver site 155 for 
distribution to the homes. 

At the reception end, the composite digital interactive signals enter a 
receiver 155 where the signals are demultiplexed, downconverted, demodulated 
5 and then passed to a cable distribution system that directs the signals to the 
homes. Although a cable distribution system 150 is the preferred transmission 
media to the homes, the digital signals may also be distributed by any 
conventionally known technique including satellite 15 to digital satellite 
receivers 155 at the home, fiberoptics, low or high power broadcast television 175, 
10 telephone lines, cellular networks, and similar technology can be used 
interchangeably with this program delivery system. 

C. The Interactive Digital Box 

The interactive digital box 25 is shown schematically in Figure 2. 

15 Preferably, the interactive digital box is a specially adapted digital cable box 25. 
The controller 260 determines what video, audio, graphics and/or Web pages to 
display based upon the interactive commands which it receives. Based upon the 
commands, it plays the appropriate video, audio, graphics or Web page options. 
The graphics can either be created and sent from the control studio 5 or the 

20 graphical images can be created at the interactive digital box 25 based on 

instructions preferably in the interactive commands. The interactive digital box 
25 connects to a television 165 or other display monitor. Further, the interactive 
digital box 25 can be connected to a digital television 195, in which case an RF 
modulator 245 is not necessary. Each downstream transmission reaches the 

25 subscriber's house, shown in Figure 2, preferably through a tap and drop cable. 
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The user interacts with the program through the input device 20. 
Preferably, the input device 20 is a typical television remote. The user interface 
270 may be an infrared, wireless, or wired receiver that receives information from 
the input device 20. 

Regardless of the type of input device 20, user inputs can be utilized by the 
present invention immediately, or at a later time, to result in personalized 
graphics, video and/or audio presentation. For example, the present invention 
utilizes "trigger points," 500 as described below, to enable subsequent branches 
among multimedia segments during the show. 

Certain commands are sent from the control studio 5 as part of the digital 
interactive programming to facilitate the collection of user entries. These 
commands are extracted at the digital demultiplexer 210 and sent to the controller 
260 which performs the appropriate action based on the commands. Some of 
these commands are explained below. 

a. BEGIN INPUT EXTENDED 

The Begin Input command starts an input period during which the user 
may press one or more buttons to select his or her choice(s). The entry 
format of this command is set forth as follows: 

BEGIN INPUT {{N} {VALID KEYS} {DIFFERENT} {FEEDBACK} 

where: 

N The maximum number of keys that can be pressed. 

KEYS The valid keys that can be pressed. 

DIFFERENT Requires each key pressed to be different. 
FEEDBACK The feedback type provided to the viewer. 

b. BEGIN VIDEO CHOICE EXTENDED 

The Begin Video Choice begins an input period for disco mode. During 
disco mode, the video switches dynamically each time the viewer makes a 
selection. 



WO 98/41020 



PCT/US97/09045 



16 

BEGIN_VIDEO_ CHOICE MODE [KEYS] {FEEDBACK [AUDIO]} 

The disco mode allows the viewer to change channels at will, while the 
OneShot mode allows only one change of channel. 

MODE DISCO/ONESHOT 

KEYS The valid keys that can be pressed. 

FEEDBACK The feedback supplied to the user for the key(s) which are 



The mode Disco allows the viewer to change channels at will, while the 
OneShot mode allows only one change of channel. 

c. BEGIN AUDIO CHOICE EXTENDED 

The Begin Audio Choice begins an input period for disco mode. 

BEGIN_AUDIO_ CHOICE MODE [KEYS] {FEEDBACK [AUDIO]} 

MODE DISCO/ONESHOT 

KEYS The valid keys that can be pressed. 

FEEDBACK The feedback supplied to the user for the key(s) which are 
pressed. 



The Map command is used to map video or audio tracks to keys, for use in 
25 connection with the Begin Audio Choice and Begin Video Choice 

commands. If this command is omitted, Key 1 will map to Track or 
Channel 1, Key 2 to Track or Channel 2 etc. This command allows 
mapping any key to any channel. 

30 MAP KEYS [TRACKS [Tl^.TN/Tl-TN] CHANNELS [C1,...CN/C1-CN]] 

The map statement maps audio tracks or video channels to keys, to enable 
audio or video choice commands to effect changes to tracks other than the 
default tracks, which are that key 1 maps to track 1, key 2 to track 2, etc.. All 
the choice statements after a map statement will cause the tracks, video 
35 tracks or channels to be changed to those specified in the map statement. 

The acceptable numbers for the map command are from 1 to 8, for keys, 
video channels and audio channels. 



40 Other commands include those which allow for the following applications: 

(1) viewer profiling, to enable the set top box 25 to "remember" viewer 
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preferences; (2) uploading viewer responses to a central location; (3) downloading 
of text and graphics, for display using the graphics chip of the set top box 25; (4) 
the ability of the viewer to prepare his own video, based upon his selections of 
camera shots 100 and audio, which can be stored and replayed for the viewer. 
5 The interactive digital box 25 of the present invention enables seamless 

flicker-free transparent switching between the digital video signals. "Seamless" 
means that the switch from one video signal to another is user imperceptible. 
Because the video signals are running off the same clock, the interactive digital 
box 25 is capable of providing a seamless digital switch from one video signal to 

10 another signal. The program clock reference necessary for the box to make this 
seamless switch is preferably embedded in the signal header. 

As shown in Figure 2, a CPU 260 is connected to an RF demodulator 200 
and digital demultiplexer 210. The CPU 260 directs demodulation and 
demultiplexing of the proper channel and data stream to obtain the correct video 

15 signal. Seamless switching can occur with MPEG-2 compressed signals since 
there are points within the frame wherein seamless switching can occur. 
Preferably, switches occur at an "I" frame, assuming the use of MPEG-2 
compression. The selected video signal is determined either by examination of 
the user's input from user interface 270 and/or any other information or criteria 

20 (such as personal profile information) stored in RAM/ROM 265. For example, 
the RAM/ROM 265 could store commands provided within the video signals as 
discussed in U.S. patent no. 4,602,279, and incorporated herein by reference. 

The RF demodulator 200 demodulates data from the broadcast channel 
directed by the controller 260. After the data stream is demodulated, it passes 

25 through a forward error correction circuit 205 into a digital demultiplexer 210. 
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The demultiplexer 210 is controlled by the controller 260 to provide a specific 
video signal out of a number of video signals which may be located within the 
data stream on the demodulated broadcast channel. The demultiplexed video 
signal is then decompressed and decoded by decompressor/decoder 215. The 
5 video signal is synchronized by a sync add circuit 220 and a sync generator 225. 
The video signal is then buffered by a video frame buffer 230. The buffered video 
signal is modulated by a modulator 245 into a NTSC compatible signal. Such a 
modulator is not necessary if the selected signal is sent to a digital television 195. 
By using a video frame buffer 230 and delaying the viewing of a given 

10 signal, enough time is allowed for the decompressor/decoder 215 to lock onto, 
decompress, convert to analog, and wait for the resultant vertical interval of a 
second video signal. For example, assume video signal A is currently being 
processed and transferred through the circuit shown in Figure 2 and displayed. 
Based upon a user selection, the controller 260 directs the digital demultiplexer 

15 210 and RF demodulator 200 to switch to another video signal, video signal B. To 
accomplish this, the analog video from the first digital video signal, video signal 
A, complete with video sync, is fed into video frame buffer 230. This buffer 230 
can hold the full video picture for "n" number of frames after which the signal is 
output to the display. In effect, a delayed video signal A is viewed "n" number of 

20 frames after the signal has been received. When the user selects a different video 
path by means of pressing a button on a keypad or entry by other means, the 
controller 260 instructs the digital demultiplexer 210 to stop decoding signal A 
and lock onto signal B to begin decoding signal B instead of signal A. 

While this is happening, even though the decompressor/ decoder 215 is no 

25 longer decompressing video signal A, the display is still showing video signal A 
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because it is being read from the buffer 230. As soon as decompressing and 
decoding occurs, the controller 260 looks for the next vertical blanking interval 
(VBI) and instructs the video frame buffer 230 to switch to its input, rather than 
its buffered output at the occurrence of the VBI. 
5 Since the RF demodulator 200, forward error corrector 205, digital 

demultiplexer 210, and decompressor/decoder 215 require a certain time period to 
decompress and decode the video signal B frame from its data stream, the size of 
the buffer 230 has to be large enough so that this processing can take place 
without interruption during the switching of the video signals. If desired, the 

10 system may continue to use the buffer in anticipation of a future switch. By using 
the controller 260 to manipulate the fill and empty rate of the buffer 230, the 
buffer 230 may be rapidly filled with video signal B frames and then after a period 
of time will be reset and ready to make another switch to another video in the 
same manner. The buffer 230 may also be reset by skipping frames or providing a 

15 delay between sequential frame outputs for a short time in order to fill the buffer 
230. If a delay is used to maintain video signal or frame output while the buffer 
230 is being filled, a slight distortion may occur for a brief amount of time. 

Because a first video signal is always displayed as the output of the buffer 
230 after the delay, the buffered video masks the acquisition and decoding of a 

20 second video signal. As long as the buffer 230 is large enough to keep the first 
video running while the second video is being decompressed and decoded, a 
seamless switch will occur. 

While the digital interactive box 25 of Figure 2 provides video 
interactivity, audio and /or graphics interactivity is also provided. For example, 

25 if, based on the viewer profile or viewer response to query, it is determined that 
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the viewer's primary language is Spanish, then that viewer could obtain Spanish 
commentary to the football, soccer, etc. game. Alternatively, if a viewer has a 
favorite athlete, the audio can switch to an interview with the athlete during a 
segment of the broadcast. Multiple digital audio options forming a set of suitable 
5 responses to an interrogatory message can be sent as part of the composite digital 
signal. As set forth in U.S. patent no. 5,585,858, herein incorporated by reference, 
there are a number of different ways to effectively forward the necessary audio 
options for a given live event to the digital interactive box 25. With the present 
invention, it makes no difference how the audio options reach the digital 
10 interactive box 25, as long as they are available for selection and play at the 
appropriate times. 

In Figure 2, the digital demultiplexer 210 extracts the digital audio signal(s) 
and forwards them to the audio switch 250. Additional audio options are 
available from the digital audio memory 255. At certain times during the 

15 program, the data codes will identify the selection of a particular audio option 
corresponding to previous user inputs. The controller 260 calls the appropriate 
audio options from internal memory 255 or directs the audio switch 250 to select 
a predetermined audio segment received as part of the received digital signal for 
passage to the RF modulator 245 for play to the subscriber. At the end of the 

20 audio segment time period as indicated by the data codes, the controller 260 
instructs the audio switch 250 to again pick up standard audio. 

The digital demultiplexer 210 sends the extracted graphics data or ACTV 
data codes to the controller 260. The controller 260 interprets the extracted data as 
either control data, including instructions for switching between video signals, 

25 audio signals, or graphics data for on-screen display. If the data is on-screen 
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display data, the data is preferably prefixed by a command designating the data as 
on-screen display data, as opposed to control data. Further, the controller 260 also 
examines the control data for the occurrence of a header code designating the 
onset of a trigger point 500 in the program, explained below. 
5 Figure 3 shows an alternate, dual tuner embodiment for seamless 

switching between separate video signals. This embodiment presumes that two 6 
MHz channels are used, each of which comprises compressed digital video and 
audio streams. In this embodiment, the microprocessor 260 controls the selection 
of the RF channel that is demodulated by RF demodulators 200A, 200B. The 

10 demodulated data streams enter the forward error correctors 205A, 205B. At the 
output of the forward error correctors 205A, 205B the data streams are transmitted 
to the input of the digital demultiplexers 210A, 210B. 

As with the RF demodulators 200A, 200B, the digital demultiplexers 210A, 
210B are controlled by the microprocessor 260. This configuration allows the 

15 microprocessor 260 to independently select two different individual time- 
multiplexed video signals on different channels and data streams. If all the video 
signals of an interactive program were contained on a single channel or data 
stream, it would only be necessary to have a single RF demodulator 200, forward 
error corrector 205, and digital demultiplexer 210 serially connected and feeding 

20 into the two digital video buffers 230A, 230B. 

Two data streams are provided from the digital demultiplexers 210A, 210B. 
One data stream carries video information pertaining to the video signal the user 
is currently viewing. The second data stream carries the video signal selected 
based on the user's previous and /or current interactive selections from the user 

25 interface 270, as determined by the microprocessor 260. 
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The digital information on each of the two streams is buffered in digital 
video buffers 230A, 230B. The buffered signals are then decompressed and 
converted into analog signals by decompressors /decoders 215 A, 215B which 
include digital to analog converters. The decompressors 215A, 215B are preferably 
5 MPEG-2 decoders. 

A local sync generator 225 is connected to sync add 220A, 220B and frame 
sync 380A, 380B circuits. Because both streams are synchronized based on signals 
from the same local sync generator 225, each stream becomes synchronized to the 
other. In particular, the signals on each stream are frame synchronized. 
10 A vertical blanking interval (VBI) switch 335 is connected to the 

microprocessor 260 so that the input may be switched during the vertical 
blanking interval of the current stream, resulting in a seamless switch to the 
viewer. 

The embodiment of Figure 3 operates as follows. Based on user responses 
15 and control codes, it is assumed that the microprocessor 260 determines that a 
switch from video signal A to video signal C should be performed. The RF 
demodulator 200A and digital demultiplexer 210A are processing the currently 
viewed video signal, video signal A, which is progressing through the upper 
branch components. A command is issued from the microprocessor 260 to the 
20 RF demodulator 200A, 200B commanding a switch to the channel and data 

stream on which video signal C is located. The microprocessor 260 also instructs 
the digital demultiplexer 210B to provide video signal C from the received data 
stream to digital video buffer 230B. 
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At this point, the upper RF demodulator 200A and digital demultiplexer 
210A are still independently receiving and processing video signal A, which 
continues through the upper branch of the circuit. 

At a certain point, the digital decompressor /decoder 215B in the lower 
5 branch will begin filling up with video signal C frames. After video signal C is 
decompressed and decoded, it is converted into analog. A local sync generator 225 
inserts both local sync and frame sync to video signal C via sync add circuit 220B 
and frame sync circuit 380B in order to synchronize it with the currently 
displayed video signal A, which is still being provided from the upper digital 
10 video buffer 230A. At the appropriate switch point, triggered by programming 
codes supplied with each video signal A and C, the microprocessor 260 directs the 
VBI switch 335 to switch in the vertical blanking interval from video A to video 
C, at which time video C will then seamlessly appear on the computer screen. 

Digital video buffers 230A, 230B may be used in the circuit of Figure 3, but 
15 are optional. However, in an alternative embodiment the buffers would be 
required to provide a seamless switch if the Figure 3 circuit was modified to 
incorporate a single RF demodulator 200, single forward error corrector 205, and 
single digital demultiplexer 210, each with a single input and single output. In 
this alternative embodiment, the circuit cannot independently receive and 
20 demultiplex two data streams on different frequency channels. One buffer 230A 
is used to store previously received video signals, while the other buffer 230B 
quickly passes through the selected video signals. 

Based on the same assumptions above, video signal A is progressing 
through the upper branch of the circuit and it is desired to switch to video signal 
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C. However, in this alternative embodiment, the digital video buffer 230A is 
providing maximum buffering to video signal A. 

Because it is desired to switch to video signal C, the microprocessor 260 
directs the alternative circuit (containing a single RF receiver 200, single forward 
5 error corrector 205 and single digital demultiplexer 210 connected in serial), to 
receive and demultiplex the data stream on which video signal C is located, 
which may be different than that of video signal A. When video signal C is 
demultiplexed, the microprocessor 260 directs the digital video buffer 230 to 
provide minimum buffering of video signal C so that decompressor/ decoder 215 

10 may quickly decompress and decode the digital signals. After decompression and 
decoding, video signal C is synchronized with video signal A. At this time, video 
signal A is read for display from digital video buffer 230A. The upper digital 
video buffer 230A must be large enough to provide video frames for output 
during the time it takes the RF demodulator 200 and digital demultiplexer 210 to 

15 switch to video signal C and the time required for decompression, decoding, and 
synchronization of video signal C. 

When video signal C is synchronized with video signal A, the 
microprocessor 260 directs VBI switch 335 to switch from video signal A to video 
signal C in the vertical blanking interval of video signal A, thereby providing a 

20 seamless and flicker-free switch. 

At this time, digital video buffer 230 will begin to utilize maximum 
buffering by altering its fill/empty rate as described above with respect to the 
Figure 3 embodiment. When adequate buffering is achieved, a switch to another 
video signal may be performed in the same manner as described above. 
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Another preferred embodiment is shown in Figure 4. This embodiment 
also includes an RF demodulator 200, a forward error corrector 205, and a digital 
demultiplexer 210. However, the circuitry differs along the rest of the chain to 
the television set or monitor. In this embodiment, a memory 475 is incorporated 
5 and connected to the output of the demultiplexer for storing the compressed 
composite digital video signal. The decompressor/decoder 215 is inserted at the 
output of the compressed memory. The decompressor/decoder 215 decompresses 
the digital signal, converts the signal to analog and forwards the analog signal to 
the RF encode 245 for transmission to the monitor. Once the composite 

10 compressed digital video signal is fed into the compressed memory 475, the 
microprocessor 260 directs a pointer to be placed somewhere along the 
compressed digital video signal. Based on the placement of the pointer, different 
frames and different segments of the composite digital video signal will be read 
from memory 475 for decompression and decoding. 

15 The different video signals are distinguished from one another because 

they are labeled, preferably by headers. Assuming that video signal A has been 
selected for play on the monitor, the compressed digital memory 475 fills up with 
A frames. Assuming a switch to video signal C is desired, the microprocessor 260 
directs the RF demodulator 200 and digital demultiplexer 210 to begin filling the 

20 compressed memory 475 with video C frames. The decoder 215 pointer begins to 
move down. As soon as a sufficient number of C frames have entered the 
compressed memory 475, the pointer will then jump to the beginning of the C 
frames. The C frames are then output into the decompressor /decoder 215 where 
the digital frames are converted into an analog signal. 
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The digital video is multiplexed in a series of easily identifiable packets. 
These packets may contain full compressed frames of video (I frames) or may 
include only the differences between full frames (B frames or P frames). 

To be able to reconstruct the full video images, the decompressor/decoder 
5 215 needs to have a minimum number of I, P and B frames. The decoder 215 
needs only one I frame to decode an image. Conversely, two prior Anchor frames 
('TV and "P's") are necessary to decode B frames. In order to decode P frames, the 
decoder 215 only needs one Prior Anchor frame. When the microprocessor 
instructs the digital demultiplexer 210 to start sending packets from a different 

10 data stream there is no way to be certain that the next packet will be an I packet 
needed for decoding the second video stream. To avoid a breakup of the video 
images, which would occur if the decompressor/decoder 215 suddenly started 
receiving packets unrelated to the stream it was decoding, the microprocessor 260 
starts to fill up the memory 475 with video signal C packets until it is determined 

15 that a full sequence of I, B and P frames are available. The decoder 215 should 
receive the last bit of the last B frame in a given, GOP (Group of Pictures) before 
the switch, in order to prevent glitches when decoding. Furthermore, the last B 
frame of the GOP must only be backward predicted, not forward predicted or 
bidirectional predicted. As soon as the valid sequence is in memory 475 the 

20 microprocessor 260 moves the memory read pointer to the start of a valid 
sequence of C video signal packets so that the decompressor/decoder 215 can 
successfully decode the C signals. This results in a seamless switch from video 
signal A to video signal C. 

This embodiment requires a data channel for enabling a synchronous 

25 switch between a first video stream and a second video stream. This data channel 
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comprises the ACTV codes which link together the different program elements 
and information segments on the different video signals. In addition, the data 
channel also comprises synchronization pulses and a time code to signify to the 
pointer the proper time to skip from a memory location representing one video 
5 signal to a memory location representing another video signal in order to enable 
a seamless switch. 

The microprocessor 260 reads the data signal from the digital 
demultiplexer 210 and communicates pertinent data to the sync add circuit 220, 
which is connected to sync generator 225. The microprocessor 260 is then able to 

10 synchronously communicate with the memory 475. 

The time code sent will identify the timing for one picture, as well as for 
multiple pictures, and will lock the different pictures together. This is done 
through the use of similar clocks at both the transmission end and the receiver. 
A time code is used in order to keep the two clocks at both the transmission and 

15 receive end synchronously connected to one another. Once the clocks at both 
ends are working synchronously, each of the multiplexed video streams must be 
synchronized to the clocks. In order to synchronize the multiplexed video stream 
to the clocks, each of the individual channels must be referenced to a common 
reference point and must be identified. 

20 In the preferred embodiment, a packet header would be incorporated into 

the transport layer of the MPEG signal to identify the various channels. The 
packet header will also include information as to where to insert the vertical 
blanking interval. In MPEG, the vertical blanking interval is not transmitted 
from the headend. Therefore, the vertical blanking interval must be generated 

25 locally. The packet header eye will identify at what time the vertical blanking 
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interval is in existence in order to effectuate a seamless switch between analog 
pictures. 

In summary, the combination of clock and the information embedded in 
either the transport layer of MPEG or in a separate packet on a separate data 
5 channel effectuates the linking between each video signal and a corresponding 
time point. The data channel also includes information designating when all the 
various video signals will be in synchronism with one another. It is at these 
points that the microprocessor 260 may direct the pointer to skip from one 
location to another location, at a time (such as during the VBI) when a seamless 
10 switch will result. 

D. Trigger Points 

Interactivity is further enhanced in the digital interactive embodiments 
through the application of trigger points 500 scattered at various predetermined 

15 times throughout the program, a timeline representation of which is shown in 
Figure 5. The trigger points 500 correspond to times when interactive events are 
scheduled to take place during the live sporting event 10. These interactive 
events could be the selection and playing of video, audio segments, the display of 
graphics or display of Web pages accessed from Internet Web sites 170. For 

20 example, when a viewer's favorite baseball player is at bat, graphics showing past 
or current performance statistics of the player can be overlaid on the screen while 
excerpts from an interview with the player can be played for the viewer. While 
the choice of particular video, audio or graphics is still dependent on viewer 
selections, the viewer selections in response to displayed graphical interrogatory 

25 messages are preferably made during a period at the onset of the program or 



WO 98/41020 PCT/US97/09045 
29 

when a viewer first tunes into the program. Alternatively, interrogatories are 
not necessary if the switches are based on the viewer profile stored in memory 
265. These viewer selections are then utilized as inputs to macros called up at 
later times during the program by the controller 260 upon the occurrence of the 
5 trigger points 500, identified to the interactive computer by unique codes 
embedded in the video signal. 

The trigger points 500 correspond to the times when the conventional 
program content can be altered and personalized for the viewers. The 
programmer can place the trigger points 500 at any time throughout the program. 

10 Since the trigger points 500 are unknown to the subscriber, the subscriber does not 
know when they will receive a personalized message. In other words, an 
interactive response can either immediately follow a corresponding user 
selection made to an interrogatory message or occur at a later time corresponding 
to a trigger point 500, or any combination of the two. Of course, timing of the 

15 interactive events should correspond to suitable times in the program where 

branching to interactive elements is sensible and does not clash with the program 
content of the conventional video still displayed on the television 165 or other 
display monitor. 

At the onset of a trigger point 500, the controller 260 will select one of 
20 several possible audio (or video or graphic display) responses for presentation to 
the subscriber. As mentioned above and shown in figure 5, some of the 
responses may comprise a branch to either a video segment, graphics and/or 
audio segments. 

In combination with the use of trigger points 500, the present invention 
25 allows for the viewer to select certain options at the onset of the program to suit 
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the viewers' preferences. For example, if the program broadcast is a live sports 
event 10, at an early trigger point 500, the viewer could be queried as to whether 
the viewer would prefer to receive audio in English, Spanish, French, or perhaps 
hear the local announcer instead of the network announcer. Upon the viewer 
5 selection, the CPU 260 directs a branch to the appropriate interactive segment. 

Each trigger point 500 is identified preferably through the broadcast of 
ACTV codes sent as part of the composite interactive program signal. The codes 
preferably include, at a minimum, the following information: (1) header 
identifying the occurrence of a trigger point 500; (2) function ID (e.g., selection of 
10 audio or graphics responses, etc.); and (3) corresponding interrogatory message(s) 
or particular viewer characteristic or habit based on viewer profile. The first bit 
sequence simply identifies to the controller that a trigger point 500 is about to 
occur. The function ID designates the macro or other set of executable 
instructions for the controller 260 to read and interpret to obtain the desired 
15 result, e.g., a selected video and/or audio response. 

Upon extraction of the codes by the data decoder, the controller 260 reads 
and interprets the codes and calls from memory 265 a particular user selection(s) 
designated by the trigger point 500 codes. The user selections correspond to 
subscriber answers to a series of interrogatory messages preferably presented at the 
20 beginning of the program. After obtaining the appropriate user selection(s), the 
controller 260 reads and performs the executable instructions using the user 
selection(s) as input(s) in the macro algorithm. The result of the algorithm is 
either a selected video stream, audio and /or selected graphics response. The 
video/ audio response can be called from memory 265 if it is prestored, called 
25 from external data storage, or the controller 260 can command the switch to 
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branch to the particular video audio stream if the response is broadcast 
concurrently with the trigger point 500. After the selected video/audio response 
is played to the subscriber, the switch branches back to the standard program, 
shown at time t g in Figure 5. 
5 As mentioned above, a series of interrogatory messages are preferably 

presented when the subscriber begins watching the interactive program. These 
interrogatory messages can be presented in any one of three ways. First, the 
interrogatory messages can be presented as graphics displays overlaid by the 
interactive computer workstation onto a video signal, wherein the graphics data 
10 is sent in the vertical blanking interval of the composite interactive signal, or 
alternatively stored on the hard disk or external storage. Second, the 
interrogatory messages are presented as graphics displays as discussed above, 
except the graphics data comes from local storage, external data storage (e.g., CD 
ROM, cartridge, etc.), or a combination of data in the VBI and data called from 
15 either local or external data storage. Third, graphics data can be presented in the 
form of user templates stored at the interactive computer workstation. 

User selections corresponding to answers to the n successive interrogatory 
messages are received by the remote interface 270 at the beginning of the show, 
stored in memory 265 and used throughout the show at the appropriate trigger 
20 points 500 to subtlety change program content as the show progresses. Preferably, 
each interrogatory has a set of possible answers. Next to each possible answer will 
be some identifier corresponding to a label on a key on the user interface. The 
subscriber depresses the key corresponding to their answer selection. This 
selection is decoded by the remote interface 270 and controller 260, stored in 
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memory 265, preferably RAM, and used later as required by an algorithm 
designated at a trigger point 500. 

E. Internet 

5 In addition to the central studio serving as a source of interactive option 

responses, the Internet can be used as a source of personalized information for 
interactive responses. 

As discussed above, the video programming is preferably created at a 
centralized location, i.e., the control studio 5 as shown in Figure 1, for 

10 distribution to subscribers in their homes. Referring to Figures 1 and 6, in a 

preferred method, the operator at the control studio 5 must designate certain Web 
pages to correspond to one of the program options, such as audio and /or graphics 
options, using control PC 135 and URL encoder 600. Instead of encoding the 
actual content of the options at the control studio 5, as with the audio, Web 

15 address identifiers, i.e., Uniform Resource Locators (URLs) are encoded and sent 
as part of the data codes from the control PC 135 to the digital multiplexer 140. 
After multiplexing, the URLs are sent as part of the program signal 615, as 
described above. Preferably, the URLs, like the various audio and graphics 
options, have associated time stamps which indicate to the remote digital set top 

20 boxes 25 when, during the video program, to display the particular Web pages 
addressed by the URLs, the selection and display of which is preferably made as a 
function of viewer responses or viewer profile. 

Preferably, each digital set top box 25 has an Internet connection 160 created 
concurrently with the cable connection. The Internet connection 160 can be via 

25 high-speed line, RF, conventional modem. The digital set top box 25 has Internet 
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access 160 via any of the current ASCII software mechanisms. In a preferred 
embodiment, in the interactive digital set top box 25, the digital demultiplexer 210 
extracts the URLs along with the other data codes. In an alternative embodiment, 
a local URL decoder 605 at the user site extracts the URLs. 
5 In a preferred embodiment, a JAVA enabled browser as well as specialized 

software for performing part of the method of the present invention are installed 
on the interactive digital set top box 25. The JAVA enabled browser allows the 
interactive digital set top box 25 to retrieve the Web pages and is preferred 
software, since it is platform independent, and thus, enables efficient and flexible 

10 transfer of programs, images, etc., over the Internet. The specialized software acts 
as an interface between the video programming and the Internet functions of the 
present invention. The processor and software interprets these URLs and directs 
the JAVA enabled browser to retrieve the particular relevant Web pages, and 
synchronizes the retrieved Web pages to the video content for display on the 

15 television monitor 165 at the appropriate times. 

In the present invention, the viewer also has the capability to link to a 
channel website at will. For example, if a viewer is interested in purchasing a 
product described in an advertisement, by merely clicking on a button on their 
remote 20, the producer's Website could be accessed by Internet connection 160 

20 and displayed to the viewer. The viewer could then either obtain more 

information about the product or order the product, if desired. As described 
above, this application is possible by sending the URL associated with the 
producer's Website to the digital cable boxes 25 as part of the interactive program. 
Upon selection by the viewer, the web browser, located either in the digital set-top 
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box 25 or externally in a connected PC 610, can retrieve the Web pages. The 
specialized software then synchronizes the Web pages for video display. 

F. Memory 

5 The interactive digital set top box 25 of the present invention also has the 

advantage of remembering subscriber responses and using these responses in 
choosing a video/audio response, and/or graphics interrogatory message, to 
present to the student. Memory branching is a technique of the present 
invention where the algorithm assembles video/audio responses and graphics 

10 interrogatory messages according to the current and previous user inputs. 
Memory branching is accomplished by linking video/audio streams and/or 
successive graphics interrogatory messages together in a logical relationship. In 
this scheme, the interactive digital set top box 25 contains logic (preferably, in the 
software algorithm) and memory 265 to store previous subscriber selections and 

15 to process these previous responses in the algorithm to control future 

video /audio stream selection, as well as future graphics message selection. 

G. Digital Viewer Profiles 

In a preferred embodiment, the interactive digital cable box 25 can have a 
20 "viewer profile" stored in its memory 265. Alternatively, the accumulated profile 
of viewer characteristics and /or habits can be stored at the control studio 5 or 
cable headend. If the profile statistics are accumulated at some central location, 
they can also be sent to each viewer's home. On the other hand, if accumulated 
in memory 265 at each of the set top terminals 25, the data could be sent to the 
25 central location for storage and dissemination by way of the digital back channel. 
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Decisions regarding personalized advertising and viewing can then be made for a 
viewer or class of viewers based on the accumulated parameters. 

The "viewer profile" preferably contains characteristics of the particular 
viewer at that subscriber location, such as sex, hobbies, interests, etc. This viewer 
5 profile is created by having the viewer respond to a series of questions. 

Alternatively, the viewer profiles could be created at a control studio 5 and sent to 
the interactive digital cable box 25. This information is then used by the cable box 
software to create a compendium of the viewer's interests and preferences ~ i.e., a 
user profile. The stored user profile would be used in place of the 

10 question/answer format, and thus, dictate the branches to interactive segments of 
interest to the viewer. 

Alternatively, the interactive cable box 25 can be programmed to create a 
user profile of each viewer based on the selections made during one of the 
interactive programs. If the profile shows that a particular viewer does not enjoy 

15 violence, the system can automatically branch to another video signal at the 

commencement of a fight in a sports program. Furthermore, such a user profile 
could be modified or enriched over time based on selections made during future 
interactive programs. For example, the 'memory' technique described above can 
be used to modify the user profile based on user response over time. 
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Event data is collected from the viewer when the following command is 
received and processed by the controller 260 in the interactive digital cable box 25: 

VIEWER PROFILE 

5 This command is used to enable events which can be utilized for profiling 

during a show. The events indicated in this command are those profiling 
events which are possible during the show. The enabled events may be 
either selected by the viewer during the show, or may be automatically 
enabled based upon viewer selections. 

10 

VIEWER_PROFILE EVENT1 / ...EVENT24 

Once the profile is created, the programming choices or interactive 
responses can be triggered based on the content of the viewer profile itself. For 

15 example, if the viewer profile suggests that the viewer is particularly interested in 
sports cars, a sports car commercial could be played for the viewer at a 
predetermined point in the program. As another application, if a viewer's profile 
indicates that the viewer is interested in cooking, whenever the viewer watches 
such a program, the user profile would trigger the interactive program to 

20 download recipes and either display such recipes on the screen or send the recipes 
to an attached printer. 

Viewer profile information can then be collected at the control studio 5 
through polling of the viewer cable boxes 25 for viewer selection data. Special 
polling software is loaded into the digital set top box 25 for performing the polling 

25 functions. Alternatively, the invention allows for the digital set top boxes 25 to 
send back data on command from the control studio 5 or periodically. The 
command to initiate an upload of viewer profile data is as follows: 
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UPLOAD EXTENDED 

This command is used to initiate the uploading of data to the central site. 

UPLOAD_EXTENDED UPLOAD_IDENTIFIER VARIABLE_NAME 
5 [PHONE_NUMBER] 

UPLOAD_IDENTIFIER numeric constant identifying 

upload. 

VARIABLE_NAME name of variable to be uploaded 

10 PHONE_NUMBER string, that consists of numbers 

only, it can be omitted, if not 
needed. 



15 Regardless of whether a polling or periodic scheme is used, the statistics and 
other user profile information is preferably sent back to the control studio 5 by 
use of the back-channel. 

H. Applications 

20 The embodiments, described above, allow for several possible applications. 

For example, in a live sports event 10, one channel could carry the standard video 
channel, with other channels carrying different camera angles 100 and /or close- 
ups of particular players. Other potential video options include instant replay, 
highlights, player statistics via graphic overlays, etc. Graphics presenting 

25 statistical information on the players can be constantly updated using the Chyron 
system. 

The provision of trigger points 500, explained above, can provide for 
seamless integration of such video options during the sporting event, based on 
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either the viewer responses to interrogatories at the beginning of the program 
and/ or on the digital viewer profile. 

Further, the viewer can become the director with the present invention. 
For example, the viewer can choose which camera angles 100 to emphasize. In a 
5 broadcast of golf, the viewer can direct whether they desire to follow a particular 
player from hole-to-hole, focus on one particularly difficult hole. In this 
manner, the viewer can customize the sporting broadcast to meet his own 
interests. 

Further, the viewer can act as a director to create their own video. During 
10 a live concert 10, for example, the viewer selects various camera angles 100 at 

different times. These selections along with a time stamp, indicating the program 
time when each selection was made, are stored in memory 265. When the 
program is played back a second time, the processor 260 will automatically direct 
branching between the video channels according to the stored selections at the 
15 time stamp. In this manner, a music video can be created by the viewer. 

The viewing experience can be further enlightening for the viewer by 
implementing games and contests during the live sporting event 10. For 
example, graphics overlays can be developed that query the viewer during the 
game. During a football broadcast, for example, viewers can be queried with such 
20 interrogatories as the following: 

What will be the next play? (RUN/PASS/KICK); 
Will the offense get the first down?; 
Will they score on this possession?; 
Pick the halftime score; 
25 Who will win? 
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Each viewer's responses can be sent back to the control studio 5 for 
tabulation of scores. Preferably, the responses are packaged at the digital cable box 
25 and transmitted to the control studio 5 via the digital backchannel upon the 
UPLOAD EXTENDED command. Alternatively, tabulation of scores can take 
5 place at the digital cable box 25 through the utilization of certain software in 
memory 265. Each correct answer can correspond to a certain number of points. 
At the end of the game, the interactive program preferably presents a graphic 
showing the viewer point total. If desired, advertisers could present special gift 
certificates for excellent performance in such games. The provision of such 
10 certificates would occur by displaying a certain code that a viewer can take to a 
store to receive the gift. In this manner, viewer interests in sports events can be 
enhanced. 

Further, the viewer has the option with the present invention to block out 
viewing of certain events. For example, if the viewer is adverse to violence 
15 during a sporting event or other type program, the system can block out such 
options from the viewer with the following data commands: 
EVENT 

This command is used to indicate occurrence of a certain event (e.g., a fight 

breaking out during a football game). 
20 EVENT EVENTJNUMBER 

EVENT_NUMBER is a numeric constant. 

ON EVENT EXECUTE MACRO 

This command is used to implement certain actions (such as an automatic 

branch to another video channel in order to block out violent event, for 
25 example), as soon as the event arrives. 
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ON_EVENT EVENT_NUMBER MACRO_NUMBER 

EVENT_NUMBER is a numeric constant. 
MACRO_NUMBER is a numeric constant. 

I. Two-Way Configuration 

The live programming system of the present invention may be operated in 
a two way configuration, as illustrated in Figure 7. In this mode, the various 
video signals are processed as previously described, being digitized and 
compressed at the control studio 5. The signals are then sent to a central 
switching station, or headend 30. 

In this embodiment, the switching between the various live digital signals 
is accomplished at the headend 30 rather than at the receiver. On the receive end, 
each digital set-top box 760 relays viewer selections back to the remotely located 
switching station 30. Preferably, the viewer selections are relayed by way of the 
digital back channel 770. However, the viewer selections may be relayed to the 
switching station 30 by any conventional means, such as two-way cable 
television, telephone or microwave transmission. The switching station 30 
receives the viewer selection and routes the desired signal to a transmitter 750 
which conventionally transmits the desired video down the appropriate digital 
cable channel for the particular viewer. 

At the central switching station 30, a demultiplexer 710 demultiplexes the 
compressed signals and places each on a separate bus channel 725. A number of 
remote control interactive switches 730, 732, 734, 736 are connected to the video 
signal bus 725. Based on the viewer selections, an algorithm stored in memory 
265 and under processor 260 control at the central switching station 30, a digital 
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seamless switch is made and the selected video, audio and/or graphics are 
forwarded to the viewer home for display. 

Such a two-way embodiment could be implemented in a video dial tone or 
video server system. In such a system, only a single video channel 755 is 
5 necessary for each home. Once the viewer selection is received at the server site 
at the cable headend 30, a switch is made to the appropriate video stream and this 
stream is sent on the single channel 755 to the home. 

Alternatively, it may be desirable to transmit an interactive sporting event 
over a single telephone line. When the viewer enters a selection on their remote 

10 20, a signal is sent by way of the telephone line to the central switching station 30 
which routes the desired signal of the interactive program over the user's 
telephone line so that a single link handles both the interactive choice being 
made at the receiver and the transmission of that choice from the headend 30 
where the actual switching takes place in response to the interactive selection 

15 made at the receiver. 

The two-way link between the viewer and switching station 30 may be 
used for other purposes. For example, demographic data may be transferred from 
the viewer to the broadcast network for commercial purposes, such as targeted 
advertising, billing, or other commercial or non-commercial purposes. 

20 While the present invention has been described primarily with respect to 

live events, and in particular sporting events, it has equal potential for enhancing 
content in other program categories. A viewer can become their own director of a 
murder mystery or other drama. By entering responses to displayed questions at 
the initiation of or during the show, the program will branch to alternative 

25 video /audio segments as a result of the user selections. In this manner, different 
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viewers with different selections may end up with a different murderer at the 
conclusion of the broadcast. 

Using the foregoing embodiments, methods and processes, the interactive 
multimedia computer maximizes personalized attention and interactivity to 
5 subscribers in their homes in real time. Although the present invention has 
been described in detail with respect to certain embodiments and examples, 
variations and modifications exist which are within the scope of the present 
invention as defined in the following claims. 
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CLAIMS 

We claim: 

1. A live interactive digital programming system, comprising: 

a viewer television reception system for receiving live interactive 
programming, the live interactive programming comprising a plurality of 
digitally compressed video, audio, branching codes and graphics signals, the 
reception system comprising: 

a viewer interface for receiving viewer entries; 

a microprocessor, connected to the viewer interface, for selecting one 
of the video and audio signals and directing a seamless switch to the 
selected video and audio signals at a predetermined time, the selection of 
the video and audio signals and the predetermined time of each selection a 
function of the branching codes and the received viewer entries; 

a demultiplexer, for demultiplexing the selected video and audio 
signals; 

a decompressor/ decoder, connected to the demultiplexer for 
decompressing the demultiplexed selected video and audio signals; 
a means for displaying the selected video signal; and 
a means for playing the selected audio signal. 

2. The live interactive digital programming system of claim 1, wherein the 
plurality of digitally compressed video signals corresponds to different 
predetermined camera angles of an event. 
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3. The live interactive digital programming system of claim 1, wherein the 
microprocessor selects one of the graphics signals at a predetermined time, the 
selection of the graphics signal a function of the branching codes and the received 
viewer entries, and further comprising a means, connected to the microprocessor, 
for presenting the selected graphics signal on the display means. 

4. The live interactive digital programming system of claim 1, wherein the 
display means presents at least one interrogatory to the viewer, the content of the 
interrogatory involving program options, and the viewer entries correspond to 
collected entries from the viewer via the viewer interface in response to the 
interrogatories. 



5. A live interactive digital programming system, comprising: 

a viewer television reception system for receiving live interactive 
programming, the live interactive programming comprising a plurality of 
digitally compressed video, audio, branching codes and graphics signals, the 
reception system comprising: 

memory, for storing a viewer profile; 

a microprocessor, connected to the viewer interface, for selecting one 
of the video and audio signals and directing a seamless switch to the 
selected video and audio signals at a predetermined time, the selection of 
the video and audio signals and the predetermined time of each selection a 
function of the branching codes and the stored viewer profile; 

a demultiplexer, for demultiplexing the selected video and audio 
signals; 



WO 98/41020 PCT/US97/09045 
45 

a decompressor/ decoder, connected to the demultiplexer for 
decompressing the demultiplexed selected video and audio signals; 
a means for displaying the selected video signal; and 
a means for playing the selected audio signal. 

6. The live interactive digital programming system of claim 5, wherein the 
plurality of digitally compressed video signals correspond to different 
predetermined camera angles of an event. 

7. The live interactive digital programming system of claim 5, wherein the 
microprocessor selects one of the graphics signals at a predetermined rime, the 
selection of the graphics signal a function of the branching codes and the viewer 
profile, and further comprising a means, connected to the microprocessor, for 
presenting the selected graphics signal on the display means. 

8. A live interactive digital programming system, comprising: 

a viewer television reception system for receiving live interactive 
programming, the live interactive programming comprising a plurality of 
digitally compressed video, audio, branching codes, and one or more uniform 
resource locators specifying one or more Internet addresses of related Internet 
information segments obtained from Web sites on the Internet, the reception 
system comprising: 

a viewer interface for receiving viewer entries; 
a means, connected to the viewer interface, for processing 
comprising: 
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means for selecting one of the video and audio signals and 
directing a seamless switch to the selected video and audio signals at 
a predetermined time, the selection of the video and audio signals 
and the predetermined time of each selection a function of the 
5 branching codes and the received viewer entries; 

a means for decoding the uniform resource locators to 
determine the specified Internet addresses; 

a means, connected to the decoding means, for retrieving the 
one or more Internet information segments residing at the 
10 determined Internet addresses; and 

a means for presenting the video and audio signals, and Internet 
information segments. 

9. The live interactive digital programming system of claim 8, further 
15 comprising: 

a demultiplexer, for demultiplexing the selected video and audio 
signals; and 

a decompressor/ decoder, connected to the demultiplexer, for 
decompressing the demultiplexed selected video and audio signals. 

20 

10. The live interactive digital programming system of claim 8, wherein the 
plurality of digitally compressed video signals correspond to a different 
predetermined camera angle of an event. 
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11. The live interactive digital programming system of claim 8, wherein the 
presenting means displays at least one interrogatory to the viewer, the content of 
the interrogatory involving program options, and the viewer entries correspond 
to collected entries from the viewer via the viewer interface in response to the 

5 interrogatories. 

12. The live interactive digital programming system of claim 8, wherein the 
live interactive programming further comprises a plurality of graphics signals 
and the selecting means selects one of the graphics signals at a predetermined 

10 time, the selection of the graphics signal a function of the branching codes and 
the viewer profile, and further comprising a means, connected to the 
microprocessor, for presenting the selected graphics signal on the display means. 

13. A system for providing live interactive digital programming, comprising: 
15 a means for receiving video signals from a plurality of video cameras, one 

or more of the cameras relaying a different predetermined view of a live event; 

a means for producing one or more audio signals corresponding to the live 

event; 

a means for generating one or more graphics signals; 
20 at least one digital compression device, connected to the receiving and 

producing means, for digitally compressing the video, graphics and audio signals; 

a means for processing, connected to the compression device, wherein the 
processing means creates a set of data commands which link together the various 
audio, graphics and video signals, the data commands including branching 
25 commands; 
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a digital multiplexer, connected to the digital compression device, for 
multiplexing the video, graphics and audio signals, and the data codes into a 
combined digital program stream; and 

a means for transmitting the combined digital program stream. 

14. A method for providing live interactive digital programming, comprising 
the steps of: 

obtaining video signals from a plurality of video cameras, one or more of 
the cameras relaying a different view of a live event; 

producing one or more audio signals corresponding to the live event; 

creating one or more graphics signals; 

receiving the video and audio signals in a control studio; 

digitally compressing the video, graphic and audio signals; 

producing a set of data codes corresponding to the programming, the data 
codes including branching commands; 

digitally multiplexing the video, graphics and audio signals, and the data 
codes into a combined digital program stream; 

transmitting the combined digital program stream; 

receiving the combined digital program stream at a receive site; 

re-transmitting the combined digital program stream on a digital cable 
television distribution system; 

receiving the combined digital program stream at one or more viewer 

sites; 

gathering viewer specific information; 
processing the data commands; 
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digitally demultiplexing the video and audio signals resulting in a first 
video and audio signal, the first output video and first audio signal selected based 
on the data commands and gathered viewer specific information; 

instructing the digital demultiplexer to commence demultiplexing a 
second video and second audio signal, the second video signal and second audio 
signal selected based on the data commands and gathered viewer specific 
information; 

seamlessly switching from the first to the second video signal; and 
displaying the second video signal on a screen. 

15. The method of claim 14, further comprising the steps of: 

creating a viewer profile with the gathered viewer specific information; 
wherein selecting the video and audio signals are based in part on the 
viewer profile. 

16. The method of claim 14, wherein the step of gathering viewer specific 
information comprises the steps of: 

displaying at least one interrogatory to the viewer, the content of the 

interrogatory involving program options; 

collecting entries from the viewer in response to the interrogatories; and 
wherein the selection of video or audio signals is based in part on the 

collected viewer entries. 



WO 98/41020 PCT/US97/09045 
50 

17. A method for providing live interactive digital programming, comprising: 
receiving live interactive programming, the live interactive programming 
comprising a plurality of digitally compressed video, audio, branching codes, and 
one or more uniform resource locators specifying one or more Internet addresses 
5 of related Internet information segments obtained from Web sites on the 
Internet, the reception system comprising: 
obtaining viewer entries; 

selecting one of the video and audio signals and directing a seamless 
switch to the selected video and audio signals at a predetermined time, the 
10 selection of the video and audio signals and the predetermined time of each 
selection a function of the branching codes and the obtained viewer entries; 

decoding the uniform resource locators to determine the specified Internet 
addresses; 

retrieving the one or more Internet information segments residing at the 
15 determined Internet addresses; and 

demultiplexing the selected video and audio signals; 
decompressing the demultiplexed selected video and audio signals; and 
presenting the video and audio signals, and Internet information 
segments. 



WO 98/41020 



PCT/US97/09045 




SUBSTITUTE SHEET (RULE 26) 



WO 98/41020 



PCT/US97/09045 



2/7 



VIDEO 
FRAME 
BUFFER 




GEN. LOCK 
CIRCUIT 


U 


CHARACTER 
GENERATOR 
OVERLAY 








L 




SUBSTITUTE SHEET (RULE 26) 



WO 98/41020 



PCT/US97/09045 



3/7 



o 



o»9 

1X1 CE O 



£2£ 

r~\ m 



QUiQ 



<Ouj 

h- uj llI 
CD Q !i 



1 



CO 



CD UJ 



SUBSTITUTE SHEET (RULE 26) 



WO 98/41020 



PCT/US97/09045 



4/7 



o w>o 















>-cc 






DCLU 






§§ 












IS 













CD 



DIGITAL 


DEMUX 






FEC 






RF 
DEMOD 



i s 

8 £ 



SUBSTITUTE SHEET (RULE 26) 



PCT/US97/09045 



5/7 



FIG. 5 



I/- PROGRAM 
START 



TRIGGER 
POINT 500 



^TRIGGER 
T POINT 500 




STANDARD AUDIO 



SELECT AUDIoT 

t ! 



VIDEO 1 



SELECT 
VIDEO ' 



I 



GRAPHICS 0 



I VIDEO 2 
_l ^ 



SUBSTITUTE SHEET (RULE 26) 



WO 98/41020 



PCT/US97/09045 




SUBSTITUTE SHEET (RULE 26) 



WO 98/41020 



PCT/US97/09045 




_JQ_ 




— 








Oco 





EC 



5Q 



do 




RF 
DEMOD. 


II 

Oco 





CO CO CO 1 CO 



SUBSTITUTE SHEET (RULE 25) 



INTERNATIONAL SEARCH REPORT 


International applicati 
PCT/US97/09045 


on No. 


A. CLASSIFICATION OF SUBJECT MATTER 
IPC(6) :H04N 7/10. 7/12 

US CL :348/6. 7, 8. 9, 10. 12. 13; 455/3.1. 3.2. 3.3. 4.1, 4.2, 5.1 
According to International Patent Classification (IPC) or to both nation 


d classification 


and IPC 




R FIELDS SEARCHED 


Minimum documentation searched (classification system followed by c 
U.S. : 348/6. 7, 8, 9, 10. 12, 13; 455/3.1. 3.2. 3.3. 4.1, 4.2, 5.1 


assification symbols) 





Documentation searched other than minimum documentation to the extent that such documents are included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practicable, search terms used) 
STN->WP!DS, PROMPT. USPATFULL 
INTERNET >YAHOO and ALTAVISTA 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 




Relevant to claim No. 


Y 


US 5,600,364 A (HENDRICKS ET AL.) 04 FEBURARY 1997, 
FIGURES 11-13 


5-7, 15, 16 


Y 


US 5,068,733 A (BENNEff EL AL.) 26 NOVEMBER 1991, 
FIGURES 1A AND IB. 


1-17 


Y 


US 5,600,368 A (MATTHEWS ET AL.) 04 FEBURARY 1997, 
FIGURES 1-8 


1-17 


Y 


BEAUDIN, S. The Web Is Not TV, Or Is It? 
HTTP://silver.skiles.gatech.edu/gallery/dzine/wbtv/. Copyright 
1996. 


1-17 


Y 


EarthWeb And ACTV Unviel HyperTV(TM) Breakthrough. 25 
MARCH 1996. 


1-17 



Further documents are listed in the continuation of Box C. | | See patent family annex. 



Sp.oUl c.Ugom. of ciud docum.nu: "T" I.Ur document publUbed .fur lh« mum.tion.1 filing d«U or priority 

. r . . , , ... d.to nd not in conflict with the .pplidion but ciud to undentmd 

t^tf ltriS r^ncT *" " C ° m " S "° d th. principle or theory undoing the motion 

•P- .„.,'„ dooum.nt oubluh.d on ™- .ft., ih. ™.»™..;„n.l r.l.n. A.i. -x " dooument of pmrtioulv rel.v«no; th. chimed invention usnot b. 

°" h " iommtM P ubluhKl m " *• »"n»««»>»l r.lu.« d.to con.id.red noi.1 or »»t b. con.ider«i to involv. « »»«t». .Up 
•L' document which m.y ihrow doubt, on priority cl.imt.) or which i> »*•» document i. uicen .Ion. 

.peci.1 r«..on (u .pecified) "V documont or p»nj<sul*r relevance; trio cl>im«l mrentioo cmnot bo 
^ ^ f ( ^ con.id.red to inyolv. ui tnvonliv. .up »h«n ^document i> 

mean, being obviou. to . penon .killed in the art 

•P- dwumentpuWUhed prior to the mumalion.1 filing d.u but l.ur than document member of the ..me patent family 


Date of the actual completion of the international search 
21 AUGUST 1997 


Date of mailing of the international search report 

280CT1997 


Name and mailing address of the ISA/US 
Commissioner of Patents and Trademarks 
Box PCT 

Washington. D.C. 20231 
Facsimile No. (703) 305-3230 


AuthJ^jffU^^ 

MATHAN J. FLYNN 
Telephone No. (703) 308-6601 



Form PCT/ISA/210 (second sheetXJuly 1992)* 



